JRTE, 41(2), 161-177 


The Interaction of Pedagogical 
Approach, Gender, Self-Regulation, 
and Goal Orientation Using Student 
Response System Technology 

Kellah M. Edens 

University of South Carolina 

Abstract 

This research compares a behaviorally based approach for using electronic student response system 
(SRS) technology with a metacognitive-oriented approach to determine effects on attendance, 
preparation for class, and achievement. Also examined are the interaction effects of pedagogical 
approach with self regulatory and motivational characteristics of students. MAN OVA analyses 
of pretest, posttest, attitudinal, and attendance data revealed no significant difference between 
approaches on achievement. A main effect was fotmd for self-regulation level and achieve- 
ment, as well as for goal orientation and class preparation! attendance. Other findings are that 
students in the operant conditioning group were significantly more likely to be anxious when 
using SRS btit were more motivated to prepare for and attend class than the metacognitive 
group. Finally, type of pedagogy and self-regulation level also interacted significantly on the 
achievement measure, with highly self-regidated students in the metacognitive group outper- 
forming students with low self-regulatory skills. (Keywords: student response systems, effective 
pedagogy, behaviorism, metacognition, self-regulation, goal orientation.) 

INTRODUCTION 

Since the introduction of the personal computer in the late 1970s and the 
subsequent emergence of a host of technological tools and gadgets, ques- 
tions have been raised about technology’s influence on students’ learning and 
academic achievement. After 25 years of research, a consistent finding is that 
technology is not the panacea for dramatically increasing standardized test 
scores. “The amount of technology available to assist teachers is staggering, and 
it’s way ahead of teacher knowledge” (Trotter, cited in Levin-Epstein, 2000, p. 
1). Teachers at all educational levels, even those who use a computer for many 
tasks, still remain fundamentally uncertain about how to use technology to 
solve real, everyday classroom problems (Coppola, 2004). Levin-Epstein (2000) 
comments, “It’s no longer a question of teaching teachers how to use technol- 
ogy; it’s about teaching them when and why to use it” (p. 1). 

As early as 1986 (Apple, 1995), however, researchers reported changes in the 
behavior of teachers and students, noting that students were taking more re- 
sponsibility for their own learning, and teachers were working more as mentors 
and less as presenters of information. New and innovative technologies and the 
related pedagogy that potentially can transform traditional classrooms to more 
learner-, knowledge-, and assessment-centered contexts should be examined 
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empirically (National Research Council, 1999). Indeed, converging evidence 
supports a relationship between effective pedagogy and technology and multiple 
desirable outcomes, including students’ adoption of learning or mastery (intrin- 
sic) goals (Roschelle, Penuel, & Abramhamson, 2004) and conceptual change 
and achievement, particularly in physics (Burnstein & Lederman, 2001; Hake, 
1998). 

Interactive electronic response systems, also termed student or personal 
response systems, classroom or audience response systems, or simply "click- 
ers," are a software/hardware technology increasingly used in K-12 and higher 
education. For example, Classroom Performance System (CPS), only one of 
numerous response systems now available to educators, has more than 3 million 
response pads in all 30 states in thousands of K-12 schools as well as more than 
800 universities and 10 foreign countries. The technology also is utilized for 
training in corporate and military settings (e-Instruction, 2006). These remote 
control devices allow the instructor to get instant feedback easily from students 
via a portable receiver. Students are shown a question and they respond by 
pressing one of eight buttons on their remote control. Responses are tallied by 
computer and feedback is available in the form of histograms and detailed re- 
ports that are presented on the computer screen. The technology is utilized in a 
variety of ways: for taking attendance, giving informal and formal quizzes, poll- 
ing student opinions, reviewing for formal assessments, providing spontaneous 
feedback, and promoting team-building among students in large classes. 

A body of research on the practical and technical issues related to implemen- 
tation and comparison of the many marketed systems has rapidly grown (Burn- 
stein & Lederman, 2003). Students self-report that they enjoy using SRS and 
indicate that they are more attentive in class and better understand course con- 
cepts (Byrd, Coleman, & Werneth, 2004; Judson & Sawada, 2002). Recom- 
mendations for ways the technology can be used interactively have been based 
on survey data and anecdotal evidence (Purchase, Mitchell, & Ounis, 2004). 
Gains in learning outcomes in classes using SRS have been reported with end- 
of-course exam or standardized achievement test as the outcome measure. Few 
of these studies, however, used a comparison group, and none placed statistical 
controls on the comparison group (Roschelle, Penuel, & Abrahamson, 2004). 

In a 2006 review of the literature of research on student response systems, 

Fies and Marshall report that SRS research generally has compared non-SRS 
traditional teaching practice with use of SRS. For example, a study from the 
Netherlands by Poulis, Massen, Robens, and Gilbert (1998) reported data from 
more than 5,000 students who took either traditional lecture courses in physics 
or courses that used a response system. The data were collected over four years; 
during that time, some instructors taught for two years without the response 
system followed by two years with it, and another group of instructors first 
taught with the response system and later stopped using it. The content of the 
courses and the exam were monitored by the investigators to make sure they did 
not change from year to year. The end-of-course exam pass rate was used as an 
outcome measure. Findings indicated a significantly higher passing score mean 
in classes using the response system as well as a smaller variance in scores. 
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In another study, Hake (1998) surveyed 6,000 students in both university 
and high school settings and compared traditional lecture courses with interac- 
tive engagement courses. Interactive engagement courses promoted conceptual 
understanding through the use of technology providing immediate feedback 
to both the students and the instructors. The investigator found 30% to 70% 
gains in the interactive engagement courses; whereas, gains below 30% were 
found in the traditional courses. 

According to Fies and Marshall (2006), SRS research that has focused on 
pedagogical theory has investigated specific issues such as timing of feedback. 
Broader topics, such as the affordances of instantaneous feedback aggregation 
on small group interactions and whole-class evaluations, also have been ex- 
amined. For example, research on SRS used in conjunction with Mazurs Peer 
Instruction (PI) model (1997) found that SRS improved student engagement 
and learning outcomes. Other research has found that the effectiveness of the 
PI feedback model is independent of the use of technology or SRS, however. 
Another instructional sequence, based on Kolb’s experiential Learning Cycle 
(Dufresne, Gerace, Leonard, & Mestre, 2000), promotes a conceptually based 
problem-solving context in which "discussion follows a spiraling pathway" (Fies 
& Marshall, 2006, p. 104). In this context, a question leads to exploration of 
another question and another question, such that the class directs which ques- 
tions are asked and discussed 

Overall, however, after their SRS literature review, Fies and Marshall (2006) 
found that "There is insufficient research on what constitutes optimal condi- 
tions of [SRS] use" (p. 101). They further note that, in general, extant research 
has focused on individual rather than group use of SRSs and seldom depicts 
how it is used for formative assessment. Empirical evidence on the influence 
of particular pedagogical uses of the technology system on achievement thus is 
limited. Moreover, the interaction effects of other variables, such as student self- 
regulation and goal orientation with different types of pedagogical approach, 
are not known. 

To address questions related to SRS pedagogy and student characteristics, this 
quasi-experimental research study investigated two pedagogical/instructional 
uses of SRS (CPS, e-instruction) in two sections of an undergraduate introduc- 
tory course. Each section had 60 participating students. The objective of the 
study was to examine student-related behaviors, such as attendance and prepa- 
ration for class and achievement associated with two instructional approaches. 
In addition, the relationships among students’ self-regulatory and motivational 
characteristics, instructional approach, and achievement were investigated. 

The research study is framed around the following research questions: 

1. Does the instructional approach of SRS use (operant conditioning vs. 
metacognitive) differentially influence student achievement? 

2. Does instructional approach interact with individual differences, spe- 
cifically, gender, level of self-regulation, and goal orientation? 

3. Are other student outcomes (level of anxiety, class preparation and at- 
tendance) influenced by the instructional approach of SRS use? 
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The research hypotheses stated below derive from the research questions 
posed above. 

H 1 : The metacognitive approach will influence student achievement to a 
greater extent than the operant conditioning approach. 

H2: Instructional approach of SRS use will interact with self-regulation 
and goal orientation on achievement. 

H3: Tevel of anxiety, attendance, and preparation for class will be influ- 
enced differentially by approach of SRS use. 

BEHAVIORISM OR METACOGNITION: IS ONE A BETTER FIT? 
Behaviorism: The Operant Conditioning Perspective 

Although touted as an emerging and innovative technology, Judson and 
Sawada report that SRSs “cannot be considered emerging technology” (2002, 
p. 176), for even in the 1960s, instructors in some college lecture halls were 
able to ask questions, and students could give informative electronic feedback. 
The behavioral perspective was prevalent during these decades, with a Skin- 
nerian theoretical framework providing the basis for immediate feedback as to 
the correctness of an answer. Research from the 1960s and 1970s, however, did 
not find a significant correlation between student academic achievement and a 
stimulus-response method of using such systems. 

More recently, as educators have sought ways to engage students actively, 
particularly in larger classes SRS has been viewed as a viable tool. Inattentive 
behaviors, even nodding off, are more prevalent in large lecture classes (Ap- 
pleby, 1990) because of the greater anonymity than in smaller classes, and SRS 
features appear to hold students accountable for classroom participation. When 
viewed through an operant conditioning lens, attentive and engaged behav- 
iors may be influenced by innovative SRS technology. These newer technol- 
ogy systems enable the display of appealing graphic representations of student 
responses. This additional immediate and particularly salient feedback not 
present in the 1960s behavioral research, may influence student achievement, 
a finding that would contradict the 1960s research. The operant conditioning 
perspective suggests that immediate, graphically rendered and visually pleasing 
feedback may serve as positive reinforcement. Moreover, students may increase 
their study behavior because of the positive reinforcement of getting ‘a good 
grade.’ Another type of reinforcement also may come into play, however. By 
definition, reinforcement is “a stimulus [the addition] or reduction or removal 
of which strengthens behavior” (Skinner, 1989, p. 127, cited in Gredler, 2005). 
If the behaviors of reading text assignments, attending and paying attention in 
class increases because an aversive stimulus is removed or reduced (e.g., fear of 
doing poorly on the quiz, looking dumb in class, or nagging of parents about 
grades), the type of reinforcement is negative. The type of reinforcement that 
may occur within the operant conditioning approach to using SRS may, there- 
fore, be either positive or negative, depending on the response of the particular 
student. 

Cognitive Information Processing: The Metacognitive Perspective 

A paradigmatic shift from behaviorism to cognitive information process- 
ing occurred in the mid 1960s, and individuals’ cognitions, rather than envi- 
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ronmental variables, were used to explain behavior. The second pedagogical 
approach of this study focuses on the idea of developing students’ metacogni- 
tion — i.e., the deliberate and conscious control of cognitive activity (Flavell, 
1985). From the information processing perspective, metacognition is the 
executive control function that enables self-monitoring and self-assessing of 
thoughts and learning. Essentially, metacognition means thinking about think- 
ing (Garner, 1987). Through a metacognitive lens, the instantaneous graphical 
feedback inherent in SRS should facilitate students’ capacity to self-monitor 
and self-assess. The use of SRS may serve as a bridge to increase students’ meta- 
cognition, for it may encourage students to assume responsibility for the associ- 
ated self-regulatory processes of monitoring, attention control, and goal setting. 

SELF-REGULATION, GOAL ORIENTATION, GENDER, AND 
TECHNOLOGY USAGE 

Self-regulated learning includes processes, many of which are metacognitive 
in nature. Self-regulation is broader than metacognition, however, for it refers 
to self-monitoring and self-assessment of more than cognition. Research about 
students’ self-monitoring and self-assessment is grounded in motivation and 
self-regulation theories and has rapidly expanded in the past several decades. 
Self-regulation is the process of active engagement of setting goals for oneself 
and engaging the behaviors and cognitive processes that lead to achieving the 
goals. Self-regulation, positively associated with achievement, includes several 
processes, such as goal setting, planning, self-monitoring, attention control, 
self-evaluation, and solicitation of help when needed (Ryan, Pintrich, & Mid- 
gely, 2001; Winne, 1995; Zimmerman, 1998; Zimmerman & Risemberg, 
1997). Ideally, as they grow older, students’ learning behavior should become 
decreasingly regulated by parents and teachers and increasingly self-regulated 
(McCaslin & Good, 1996). 

Evidence suggests that both academic goal motivational orientation and self- 
regulated learning are directly implicated in academic performance (Pintrich 
& De Groot, 1990). Achievement motivation can take several forms, depend- 
ing on the type of goal orientation of an individual. Students with an intrinsic 
or mastery goal desire to acquire new knowledge or skills, whereas those with 
extrinsic or performance goals want to appear competent to others. In addition 
to being better self-regulators than students with extrinsic goals, students with 
intrinsic goals tend to interpret failure as a sign that they should expend more 
effort, evaluate their own performance in terms of the progress they make, and 
remain relatively calm during tests. In contrast, students with extrinsic goals 
not only exhibit less self-regulated behavior, but tend to interpret failure as a 
sign of low ability, evaluate their progress in terms of how they compare with 
others, and often are quite anxious about tests (Wigfield & Eccles, 2002). 

The discussion of gender differences in achievement typically focuses on the 
very small differences between boys and girls in verbal abilities, with females 
showing slightly higher verbal ability and males showing slightly higher visual- 
spatial ability. These differences are quite small, with considerable overlap be- 
tween the two groups. In recent years, some researchers have found that these 
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differences are getting even smaller (Halpern & LaMay, 2000). When it comes 
to achievement motivation, females tend to be more concerned than males 
about doing well in school, are more engaged in classroom activities, and more 
interested in getting a college education (Bins, Steinberg, Amorosi, & Cuevas, 
1997). Females prefer tasks at which they know they can succeed, while males 
are more willing to take academic risks and challenges (Dweck, 2000). Males, 
unlike females, tend to take their failures in stride, attributing them to lack 
of effort, whereas females attribute failure to a lack of ability, which can be a 
debilitating attribution (Dweck, 2000). Despite higher grades for females than 
males, females tend to underestimate their competence, whereas males tend to 
overestimate it (Eccles, Wigfield, & Schiefele, 1998). Studies of achievement 
goal orientation provide some evidence that males are more oriented to extrinsic 
goals and less oriented to intrinsic goals than are females (Roeser, Midgely, & 
Urdan, 1996). 

Recent research has not reported notable gender differences in the use of tech- 
nology, although gender differences in perceived self-efficacy regarding complex 
computer tasks were found over a decade ago (Bush, 1995). In a 1990 study, 
when paired in a computer lab, males tended to work on the computers while 
females tended to observe (Arenz & Lee, 1990). Males and females’ self-efficacy 
for various content domains, likely shaped by gender stereotypes, influences the 
extent to which they value different domains (Bandura, Bararanelli, Caprara, 

& Pastorelli, 2001). More recently, it was reported that, in general, females do 
not like the computer game culture (AAUW, 2000). On the surface, the use of 
clickers in the classroom has a hint of computer gaming technology, i.e., they 
provide individual electronic interaction with immediate feedback and a score 
at the end of the session. 

As stated previously, the research questions for this study focus on the main 
and interaction effects of SRS usage according to type of pedagogy (use of oper- 
ant conditioning or development of metacognition), self-regulation level, goal 
orientation, and gender. Does a pedagogy based on the behavioral perspective 
of operant conditioning or a cognitive information processing view with a con- 
centration on metacognition influence achievement? Do the outcomes of these 
approaches differ according to characteristics of particular individuals, such as 
gender, self-regulation levels, and goal orientation? How are other outcomes, 
such as level of anxiety, class preparation, and attendance influenced by the way 
SRS is implemented? 

RESEARCH METHODS 

Sample. One hundred twenty students in two undergraduate introductory 
educational psychology classes participated during fall 2006 in the research 
study at a large southeastern university. This research project involved the com- 
parison of instructional techniques in an established classroom setting; thus all 
students enrolled in the class participated. On the first day of class, students 
were informed of the way that SRS would be used and completed a pretest. 
Student data also were collected during the weekly CPS quizzes, a post test 
exam, and an end of course survey (see Procedures and Instrumentation). 
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Ninety participants were female, and 30 were male, with 90 Caucasian, 24 
African American, four Hispanic, and two Asian. These demographic character- 
istics were evenly distributed across classes. Of the participants, only 2 students 
previously had used an SRS in another course. 

Course Materials. The same instructor taught both sections. The classes met 
twice a week in the early afternoon for 73 minutes. Both classes used the click- 
ers and had access to the same informational materials and documents within 
their respective classes in a Blackboard course management system. The Power- 
point presentations, video clips, case study discussions, and other activities were 
identically sequenced for the two sections. The syllabus for each section differed 
only with regard to the SRS component and requirement, as described below. 

The Behavioral ( Operant Conditioning) Group 1: The Grading and Evalu- 
ation Component of the syllabus listed all assignments, activities, exam, and the 
associated points of each. SRS quizzes counted 25% of the final grade. 

The Cognitive Information Processing (Metacognitive) Group 2: The Grad- 
ing and Evaluation Component of the syllabus listed all points associated with 
assignments, activities, and exam. SRS was listed as an important activity in 
which students could self-monitor or self-assess their reading comprehension, 
with no points associated. Performance on the weekly quizzes was informational 
and educative for the individual student and non-punitive in terms of grades. 

Procedures and Instrumentation. 

Pretest. The pretest was completed on the first day of class prior to introduc- 
tion to course documents to measure prior knowledge of course content and to 
obtain information on students’ self-regulatory skills and motivation goals. The 
pretest was designed for the study and included items based on the Motivated 
Strategies for Learning Questionnaire (MSLQ) (Pintrich, Smith, Garcia, & 
McKeachie, 1991). The MSLQ consists of a motivation section (six subscales) 
and a learning strategies section (nine subscales). The MSLQ’s 15 scales can be 
used together or individually (Pintrich et ah, 1991). Pretest items were based on 
the following scales: Intrinsic Goal Orientation Scale (a = .74), Extrinsic Goal 
Orientation Scale (a = .62), Organization Scale(a = .64), Time and Study En- 
vironment Management Scale(a = .76), Effort Regulation Scale (a = .69), and 
Metacognitive Self- regulation scale (a = .79). Multiple choice items measuring 
prior knowledge of course concepts also were included. 

CPS Quizzes. Both groups completed a total of nine SRS quizzes on Thurs- 
days at the beginning of class on material read and discussed since the SRS quiz 
administered the week before. Each quiz had 10 multiple choice items measur- 
ing comprehension and application of key course concepts. 

Posttest: Achievement. To obtain achievement data, multiple choice ques- 
tions on course content from the pretest were administered on an exam. These 
items measured comprehension and application of course concepts. 

End of Course Survey. Students completed an online survey at the end of 
the course reporting attitudinal and affective responses to using SRS technology. 
To obtain delayed recall achievement data, multiple choice items from the pre- 
test and exam were also included. These items were slightly modified to reduce 
the practice effect from the posttest achievement measure. 
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ANALYSIS AND RESULTS 

Data were analyzed in SPSS using MANOVA. The multiple choice item pre- 
test measuring prior knowledge was not used as a covariate as no variability was 
found among students. For MANOVA, the assumption that the error variance 
of the dependent variable is equal across groups was met (i.e., Levene's Test of 
Equality of Error Variances was not significant (Exam 1 , p =.37; Anxiety, p = 

.76; Preparation and Attendance, p = .68). Mean scores and standard deviations 
on the dependent measures are reported in Table 1 for type of pedagogy and 
self-regulation level. As shown in Table 2, no significant main effects exist for 
type of pedagogy on achievement on the Exam, F(l, 119) = 1.318, p = .254. A 
main effect for type of pedagogy on level of anxiety and class preparation and 
attendance was found for Preparation and Attendance, F(l, 1 14) = 23.43 , p 
<.001, and for Anxiety, F(l, 110) =12.84 p<. 01. Table 1 shows that students 
in the operant conditioning group were more motivated to attend and prepare 
for class at a statistically significant level than the metacognitive group. In ad- 
dition, the operant conditioning group was more likely to be anxious than the 
other group when using SRS. Shown in Table 2, no main effects existed for 
gender or goal orientation on achievement, anxiety level or class preparation 
and attendance. A main effect on achievement for level of self-regulation was 
found F( \ , 110) = 6.83, p < .01, with Table 1 displaying the unexpected finding 
that students with low self-regulation scored higher than highly self-regulated 
students. 

As displayed in Table 3 (page 170), interaction effects also were examined. No 
interaction effect was found between type of pedagogy and gender, although 
level of anxiety and classroom preparedness and attendance approached a 
significant level. Several two-way interaction effects were found among other 
factors. Type of pedagogy and self-regulation level interacted significantly on 
the achievement measure F(l, 110) = 14.32,/) = .001. Highly self-regulated stu- 
dents in the metacognitive group scored significantly higher than those in the 


Table 1: Mean Score and Standard Deviation of Achievement, Attendance 
and Preparation, and Anxiety Dependent Measures by Type of Pedagogy 
and Level of Self-Regulation 


Dependent Measures 



Achievement 

Attendance/ 


Source 


(Exam 1) 

Preparation 

Anxiety 



Mean(SD) 

Mean(SD) 

Mean(SD) 

Type of 
Pedagogy 

Operant 

Conditioning 

13.4(2.3) 

6.8 (1.0)* 

3.0(.6)* 

Metacognitive 

13.6(1.9) 

5. 3(1.2)* 

2.4(,7)* 

Level of 

Low 

13.7(2.1)* 

6. 0(1. 3) 

2.6177) 

Self-Regulation 

High 

13.0(1.7)* 

6.02(1.4) 

2.7 (.67) 


* Means are significantly different between groups, p<. 01 
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Table 2: Main Effects of Type of Pedagogy, Gender, Level of Self- 
Regulation, and Goal Orientation on Achievement, Anxiety, Preparation, 
and Attendance 


Source 

Dependent 

Variable 

Type 111 
Sum of 
Squares 

df 

Mean 

Square 

F 

Sig. 

Partial 

Eta 

Squared 


Exam 

4.762 

1 

4.762 

1.318 

.254 

.014 

Type of 

Level of Anxiety 

6.310 

1 

5.430 

12.838* 

.001 

.174 

Pedagogy 

Preparation and 
Attendance 

40.056 

1 

32.341 

23.434* 

.000 

.278 


Exam 

2.535 

1 

2.535 

.702 

.404 

.007 


Level of Anxiety 

.399 

1 

.181 

.428 

.515 

.007 

Gender 

Preparation and 
Attendance 

2.061 

1 

1.207 

.875 

.353 

.014 


Exam 

24.704 

1 

24.704 

6.837* 

.010 

.066 

Self- 

Level of Anxiety 

.300 

1 

.217 

.513 

.476 

.008 

Regulation 

Level 

Preparation and 
Attendance 

2.571 

1 

.281 

.203 

.654 

.003 


Exam 

4.590 

2 

2.295 

.635 

.532 

.013 

Goal 

Level of Anxiety 

.012 

1 

.657 

1.553 

.220 

.048 

Orientation 

Preparation and 
Attendance 

7.398 

1 

.922 

.668 

.516 

.021 


*indicates significant effect 


same group with low self-regulatory skills and highly self-regulated students in 
the operant conditioning group. 

Gender and goal orientation also interacted significantly on the Exam F{\, 
110)= 7.45, p = .001, with males having an extrinsic goal orientation signifi- 
cantly scoring higher than intrinsic goal oriented males and extrinsically goal 
oriented females. Intrinsic goal oriented female students, however, scored higher 
than extrinsic goal oriented males, but not at a significant level. Gender and 
self-regulation also interacted significantly, .F(l, 110) = 7.9, />=.006, with males 
with low self-regulation scoring better than males with high self-regulation. 

Also shown in Table 3, three-way interactions were detected, which clarify 
these two-way effects. Significant interaction effects of level of self-regulation, 
goal orientation, and gender on the exam were found, F( 1, 110) = 3.67, p < 

.05. Males with high extrinsic goals significantly outperformed intrinsic goal 
oriented males. Table 4 (page 171) displays the mean scores on the Exam by 
gender, goal orientation, and self-regulation. As shown in Table 4, these high 
extrinsic goal males outscored high self-regulated males with an intrinsic goal 
orientation. High extrinsic goal male students with high self-regulation also 
outscored both intrinsic and extrinsic goal oriented female students, but not at 
a significant level. 

Table 5 (page 171) shows that males in the operant conditioning group with 
low self-regulation scored significantly higher than highly self-regulated male 
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Table 3: Interaction Effects of Type of Pedagogy, Gender, Self- Regulation, 
and Goal Orientation on Achievement, Anxiety, Preparation, Attendance 


Type III 



Dependent 

Sum of 


Mean 



Partial Eta 

Source 

Variable 

Squares 

df 

Square 

F 

Sig. 

Squared 


Exam 

.374 

1 

.374 

.104 

.748 

.001 

Type of 

Level of 







Pedagogy * 

Anxiety 

1.454 

1 

5.048 

10.541 

.002 

.136 

Gender 

Preparation and 
Attendance 

4.617 

1 

26.620 

18.960 

.000 

.221 

Type of 

Exam 







51.743 

1 

51.743 

14.321* 

.000 

.13 

Pedagogy * 
Self- 

Regulation 

Level 

Level of 
Anxiety 

Preparation and 

1.310 

1 

.328 

.686 

.411 

.010 

Attendance 

.394 

1 

1.892 

1.348 

.250 

.020 


Exam 

18.457 

1 

18.457 

5.108* 

.026 

.051 

Gender * 
Type of 
Pedagogy * 
Self RegLevel 

Level of 
Anxiety 

Preparation and 

1.953 

1 

1.953 

4.078 

.047 

.057 

Attendance 

1.479 

1 

1.479 

1.053 

.308 

.015 


Exam 

53.839 

2 

26.920 

7.451* 

.001 

.134 

Gender * 

Level of 







Goal 

Anxiety 

.632 

2 

.316 

.659 

.521 

.019 

Orientation 

Preparation and 
Attendance 

4.199 

2 

2.099 

1.495 

.232 

.043 


Exam 

28.532 

1 

28.532 

7.897* 

.006 

.076 

Gender * 

Level of 







Self- 

Regulation 

Anxiety 

3.100 

1 

.011 

.024 

.879 

.000 

Level 

Preparation and 
Attendance 

4.218 

1 

.085 

.061 

.806 

.001 


Exam 

26.705 

2 

13.353 

3.696* 

.028 

.071 

Gender * Self 
Regulation 
Level * Goal 

Level of 
Anxiety 

1.318 

1 

1.318 

2.753 

.102 

.039 

Orientation 

Preparation and 
Attendance 

.291 

1 

.291 

.207 

.650 

.003 

* Indicates a significant level 







students in both pedagogical groups. The operant conditioning group males 

with low self 

•regulation also outscored all females, although not at a significant 


level. 


WHAT DOES ALL OF THIS MEAN? 

The first research question was: Does the instructional approach of SRS use 
differentially influence student achievement? Findings did not suggest that one 
approach to using SRS was better than the other in terms of student achieve- 
ment. Students’ understanding and ability to apply course concepts were equal 
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Table 4: Mean Scores on Exam by Gender, Level of Self-Regulation, and 
Goal Orientation 


Gender 

Self-Reg Level 

Goal Type 

Mean 

Std. Error 



Intrinsic 

12.500* 

1.344 


High Self Reg 

Both 

11.417 

.776 



Extrinsic 

14.500* 

1.344 

Male 


Intrinsic 

13.933 

.694 


Low Self-Reg 

Both 

14.167 

.776 



Extrinsic 

13.750* 

.950 



Intrinsic 

14.188 

.475 


High Self-Reg 

Both 

14.056 

.501 



Extrinsic 

11.917 

.726 

Female 


Intrinsic 

14.183 

.506 


Low Self-Reg 

Both 

13.364 

.405 



Extrinsic 

12.952 

.656 


* Means are significantly different between groups. 


Table 5: Mean Scores on Exam by Gender, Type of Pedagogy, and Level of 
Self- Regulation 


Gender 


Type of Pedagogy 


Self-Reg 

Level 


Mean 

Std. Error 


N 


N 


N 



Male 

32 

Soc. Cog. 

19 

High 

Low 

8 

11 

12.417* 

13.250 

.776 

.607 

Operant 

13 

High 

4 

12.500* 

.950 





Conditioning 

Low 

9 

15.100* 

.694 

Female 

88 

Soc. Cog. 

41 

High 

Low 

20 

21 

14.398 

13.360 

.478 

.501 



Operant 

47 

High 

18 

12.375 

.466 



Conditioning 

Low 

29 

13.640 

.357 


*Means are significantly different between the groups. 


in both sections, regardless of type of pedagogy. This finding was unexpected, 
as the researcher hypothesized that the approach of using SRS to develop meta- 
cognitive processes, such as self-monitoring and self-assessment, would be asso- 
ciated with higher achievement. Answers to other research questions clarify this 
finding, however. 

The second research question asked: Does pedagogical approach interact with 
individual differences, specifically, gender, level of self-regulation, and goal ori- 
entation? 
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The preliminary evidence from this study suggests that specific student char- 
acteristics, such as gender, self-regulation, and goal orientation, play a role in 
the effectiveness of SRS on achievement. As hypothesized, type of pedagogy 
and self-regulation level interacted significantly on the achievement measure 
F( 1, 110) = 14.32, with highly self-regulated students in the metacognitive 
group significantly outperforming those in the same group with low self- 
regulatory skills. Highly self-regulated students in the metacognitive group 
also scored better than highly self-regulated students in the operant condition- 
ing group. The partial Eta squared was .13, so this effect accounted for 13% 
of the variance. This finding supports research that self-regulation is positively 
associated with achievement (Ryan, Pintrich, & Midgely, 2001; Winne, 1995; 
Zimmerman, 1998). Moreover, this interaction suggests that the aim of the 
metacognitive approach, that is, to develop the cognitive processes of self-mon- 
itoring and self-assessing, is a worthwhile effort for an instructor to pursue with 
or without SRS technology. SRS technology, despite the lack of a main effect of 
instructional approach, nevertheless provides a promising way for students in 
large classes particularly to be able to develop these self-regulatory skills. 

Gender and goal orientation also interacted significantly on the Exam F( 1, 
110)= 7.45, p = .001, with males having an extrinsic (performance) goal ori- 
entation significantly outperforming intrinsic goal oriented males, and extrinsic 
goal oriented females. The partial Eta squared for gender and goal orientation 
on the achievement measure accounted for 13.4% of the overall variance. That 
extrinsic goal oriented males outscored other students is not surprising, for 
studies of achievement goal orientation provide some evidence that males are 
more oriented to extrinsic goals and less oriented to intrinsic goals than are fe- 
males (Roeser, Midgely, & Urdan, 1996). 

Gender and self-regulation also interacted significantly, F( 1, 110) = 7.9, 
p=. 006, surprisingly, with males with low self-regulation outperforming males 
with high self-regulation. This is not consistent with the self-regulation litera- 
ture. The interaction of gender and goal orientation with self-regulation, how- 
ever, provides an explanation for this finding, for extrinsic goal oriented males 
outperformed males with an intrinsic goal orientation — regardless of self-regu- 
lation level. This aligns with research findings that males are more oriented to 
extrinsic goals previously mentioned. In addition, the 3-way interaction effect 
of gender, type of pedagogy, and self-regulation level sheds light on the find- 
ing that male students with low self-regulation level significantly outperformed 
highly self-regulated students. Low self-regulation is associated with lower 
achievement, yet in the operant conditioning group, male low self-regulators 
who also had a high extrinsic goal orientation significantly outscored highly 
self-regulated males and females. In terms of practical significance, however, the 
effect size is small, with only 5% of the effect explaining the overall variance 
on the exam. Nevertheless, the operant conditioning approach appears to have 
provided a context for some males to successfully achieve, despite low self regu- 
lation. Again, this is consistent with some previous research about a likelihood 
of extrinsic goal orientation in males. 

The third research questions asked: Are other student outcomes (level of anxi- 
ety, class preparation and attendance) influenced by the pedagogical approach 
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of SRS use? Findings from this study strongly support previous research that 
found that electronic student response systems influence students’ participatory 
behavior and active engagement (Hake, 1998; Poulis et ah, 1998). The operant 
conditioning approach significantly increased class preparation and attendance 
(F = 23.43), which is consistent with previous research, but also increased stu- 
dent anxiety (F = 12.84). From a practical standpoint, however, the effect size 
was modest for anxiety. The partial Eta squared was 13.6, which means that 
type of pedagogy accounted for only 13.6% of the overall (effect + error) vari- 
ance. The effect size for class preparation and attendance approached a moder- 
ate level, with an Eta squared pf .22 — i.e., type of pedagogy accounted for 22% 
of the overall variance. It appears that having SRS count as part of the grade 
contributes moderately to students preparing for and attending class. Students 
may have been either positively or negatively reinforced as their preparing and 
attending behaviors increased in frequency. Yet several students in this group 
voiced openly that they dreaded the CPS quiz, so for at least some students, 
negative reinforcement was occurring. In general, SRS used as reinforcement 
can increase behaviors related to class participation and engagement that may 
be desirable to instructors. 

Previous research has found that extrinsic goal oriented students tend to 
evaluate their progress in terms of how they compare with others and are often 
quite anxious about tests (Wigfield & Eccles, 2002). This finding was not sup- 
ported in this study, as no significant relationship between anxiety and goal 
orientation was found. In addition, no gender differences as to the extent that 
students liked using and valued SRS as a learning tool were noted. This is con- 
sistent with recent research, which has not found notable gender differences in 
the use of technology, although in general, females do not like the computer 
game culture (AAUW, 2000). Females and males in this study were equally pos- 
itive about SRS. For females, these clickers were not perceived as an academic 
by-product of gaming culture. 

Other secondary findings also are noteworthy. Students who used SRS as a 
metacognitive tool reported that they liked using SRS significantly more than 
students in the operant conditioning group, who tended to associate its use 
with a quiz. Students in the metacognitive group also reported that they were 
able to learn from errors they made on quizzes and liked not being penalized for 
an incorrect answer. These students also reported that they would be more likely 
to use the technology in their future classrooms. 

FUTURE DIRECTIONS 

In summary, findings from this study suggest that the more effective pedagogy 
is contingent on the instructors’ goals: If the instructor seeks to foster students’ 
adoption of intrinsic or mastery goals and encourage the development of self- 
regulatory skills, both of which are positively associated with achievement, the 
use of student response technology systems should be grounded in a cognitivist 
perspective. On the other hand, if the instructor wishes to influence class prepa- 
ration and attendance, SRS should be utilized within an operant conditioning 
framework. 
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This study sought to address a deficit in SRS literature on effective pedagogi- 
cal practices, for “Ultimately, . . . the pedagogical practices of the instructor, 
not the incorporation of the technology [is] key to student comprehension” 
(Judson & Sawada, 2002, p. 167). SRS is not “a ‘magic bullet’ to educational 
woes, these systems are merely tools which can be used in any number of ways. 
It is time to move beyond anecdotes and beyond traditional classroom pedago- 
gies” (Fies & Marshall, 2006, p. 106). Research and professional development 
should continue to focus on specific pedagogical practices for student response 
technology based on individual differences, including self-regulation and mo- 
tivation goal orientation and also pedagogical practices appropriate for diverse 
populations. Certainly, practices that use SRS in group mode should be further 
examined, including “varying degrees of anonymity in response collection” (Fies 
& Marshall, 2006, p. 106). 

Much of the current research has been conducted in the areas of physics and 
biological sciences. Other pedagogical uses that promote concept development 
in other content areas are needed. In particular, effects of types and sequencing 
of questions on student conceptual development and specific ways in a content 
area to build concepts through student discussion and feedback should be in- 
vestigated. Further research related to connections the technology can make to 
broader educational and psychological literature, such as formative assessment 
through questioning and feedback, also is warranted. 

A multitude of other uses of SRS warrants empirical investigation, given the 
powerful push of individual accountability and the fact that this technology is 
becoming more robust and affordable. As research on SRS expands and more 
is known about its affordances and limitations, perhaps its use can influence 
student learning and transform traditional classrooms to more learner-, knowl- 
edge-, and assessment-centered contexts and learning environments conducive 
to fostering conceptual development and deep understanding. 
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